<!DOCTYPE html>
<html prefix="og: http://ogp.me/ns# article: http://ogp.me/ns/article# " lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>using-stochastic-gradient-descent-for-regression | 绿萝间</title>
<link href="../assets/css/all-nocdn.css" rel="stylesheet" type="text/css">
<link href="../assets/css/ipython.min.css" rel="stylesheet" type="text/css">
<link href="../assets/css/nikola_ipython.css" rel="stylesheet" type="text/css">
<meta name="theme-color" content="#5670d4">
<meta name="generator" content="Nikola (getnikola.com)">
<link rel="alternate" type="application/rss+xml" title="RSS" href="../rss.xml">
<link rel="canonical" href="https://muxuezi.github.io/posts/using-stochastic-gradient-descent-for-regression.html">
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
    tex2jax: {
        inlineMath: [ ['$','$'], ["\\(","\\)"] ],
        displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
        processEscapes: true
    },
    displayAlign: 'center', // Change this to 'center' to center equations.
    "HTML-CSS": {
        styles: {'.MathJax_Display': {"margin": 0}}
    }
});
</script><!--[if lt IE 9]><script src="../assets/js/html5.js"></script><![endif]--><meta name="author" content="Tao Junjie">
<link rel="prev" href="using-pipelines-for-multiple-preprocessing-steps.html" title="using-pipelines-for-multiple-preprocessing-steps" type="text/html">
<link rel="next" href="using-truncated-svd-to-reduce-dimensionality.html" title="using-truncated-svd-to-reduce-dimensionality" type="text/html">
<meta property="og:site_name" content="绿萝间">
<meta property="og:title" content="using-stochastic-gradient-descent-for-regression">
<meta property="og:url" content="https://muxuezi.github.io/posts/using-stochastic-gradient-descent-for-regression.html">
<meta property="og:description" content="用随机梯度下降处理回归¶








本主题将介绍随机梯度下降法（Stochastic Gradient Descent，SGD），我们将用它解决回归问题，后面我们还用它处理分类问题。









Getting ready¶








SGD是机器学习中的无名英雄（unsung hero），许多算法的底层都有SGD的身影。之所以受欢迎是因为其简便与快速——处理大量数据时这些都是好">
<meta property="og:type" content="article">
<meta property="article:published_time" content="2015-07-27T14:59:03+08:00">
<meta property="article:tag" content="CHS">
<meta property="article:tag" content="ipython">
<meta property="article:tag" content="Machine Learning">
<meta property="article:tag" content="Python">
<meta property="article:tag" content="scikit-learn cookbook">
</head>
<body>
<a href="#content" class="sr-only sr-only-focusable">Skip to main content</a>

<!-- Menubar -->

<nav class="navbar navbar-inverse navbar-static-top"><div class="container">
<!-- This keeps the margins nice -->
        <div class="navbar-header">
            <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#bs-navbar" aria-controls="bs-navbar" aria-expanded="false">
            <span class="sr-only">Toggle navigation</span>
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
            <span class="icon-bar"></span>
            </button>
            <a class="navbar-brand" href="https://muxuezi.github.io/">

                <span id="blog-title">绿萝间</span>
            </a>
        </div>
<!-- /.navbar-header -->
        <div class="collapse navbar-collapse" id="bs-navbar" aria-expanded="false">
            <ul class="nav navbar-nav">
<li>
<a href="../archive.html">Archive</a>
                </li>
<li>
<a href="../categories/">Tags</a>
                </li>
<li>
<a href="../rss.xml">RSS feed</a>

                
            </li>
</ul>
<ul class="nav navbar-nav navbar-right">
<li>
    <a href="using-stochastic-gradient-descent-for-regression.ipynb" id="sourcelink">Source</a>
    </li>

                
            </ul>
</div>
<!-- /.navbar-collapse -->
    </div>
<!-- /.container -->
</nav><!-- End of Menubar --><div class="container" id="content" role="main">
    <div class="body-content">
        <!--Body content-->
        <div class="row">
            
            
<article class="post-text h-entry hentry postpage" itemscope="itemscope" itemtype="http://schema.org/Article"><header><h1 class="p-name entry-title" itemprop="headline name"><a href="#" class="u-url">using-stochastic-gradient-descent-for-regression</a></h1>

        <div class="metadata">
            <p class="byline author vcard"><span class="byline-name fn">
                    Tao Junjie
            </span></p>
            <p class="dateline"><a href="#" rel="bookmark"><time class="published dt-published" datetime="2015-07-27T14:59:03+08:00" itemprop="datePublished" title="2015-07-27 14:59">2015-07-27 14:59</time></a></p>
            
        <p class="sourceline"><a href="using-stochastic-gradient-descent-for-regression.ipynb" id="sourcelink">Source</a></p>

        </div>
        

    </header><div class="e-content entry-content" itemprop="articleBody text">
    <div tabindex="-1" id="notebook" class="border-box-sizing">
    <div class="container" id="notebook-container">

<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="用随机梯度下降处理回归">用随机梯度下降处理回归<a class="anchor-link" href="using-stochastic-gradient-descent-for-regression.html#%E7%94%A8%E9%9A%8F%E6%9C%BA%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E5%A4%84%E7%90%86%E5%9B%9E%E5%BD%92">¶</a>
</h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>本主题将介绍随机梯度下降法（Stochastic Gradient Descent，SGD），我们将用它解决回归问题，后面我们还用它处理分类问题。</p>
<!-- TEASER_END -->
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Getting-ready">Getting ready<a class="anchor-link" href="using-stochastic-gradient-descent-for-regression.html#Getting-ready">¶</a>
</h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>SGD是机器学习中的无名英雄（unsung hero），许多算法的底层都有SGD的身影。之所以受欢迎是因为其简便与快速——处理大量数据时这些都是好事儿。</p>
<p>SGD成为许多机器学习算法的核心的另一个原因是它很容易描述过程。在本章的最后，我们对数据作一些变换，然后用模型的损失函数（loss function）拟合数据。</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="How-to-do-it...">How to do it...<a class="anchor-link" href="using-stochastic-gradient-descent-for-regression.html#How-to-do-it...">¶</a>
</h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>如果SGD适合处理大数据集，我们就用大点儿的数据集来演示：</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [5]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">from</span> <span class="nn">sklearn</span> <span class="k">import</span> <span class="n">datasets</span>
<span class="n">X</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="n">datasets</span><span class="o">.</span><span class="n">make_regression</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="mf">1e6</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"</span><span class="si">{:,}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="nb">int</span><span class="p">(</span><span class="mf">1e6</span><span class="p">)))</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">
<div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>1,000,000
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>值得进一步了解数据对象的构成和规模信息。还在我们用的是NumPy数组，所以我们可以获得<code>nbytes</code>。Python本身没有获取NumPy数组大小的方法。输出结果与系统有关，你的结果和下面的数据可能不同：</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [9]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="nb">print</span><span class="p">(</span><span class="s2">"</span><span class="si">{:,}</span><span class="s2">"</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">nbytes</span><span class="p">))</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">
<div class="prompt"></div>
<div class="output_subarea output_stream output_stdout output_text">
<pre>800,000,000
</pre>
</div>
</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>我们把字节码<code>nbytes</code>转换成MB（megabytes），看着更直观：</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [10]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">X</span><span class="o">.</span><span class="n">nbytes</span> <span class="o">/</span> <span class="mf">1e6</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">
<div class="prompt output_prompt">Out[10]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>800.0</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>因此，每个数据点的字节数就是：</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [11]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">X</span><span class="o">.</span><span class="n">nbytes</span> <span class="o">/</span> <span class="p">(</span><span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">*</span> <span class="n">X</span><span class="o">.</span><span class="n">shape</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">
<div class="prompt output_prompt">Out[11]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>8.0</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>这些信息和我们的目标没多大关系，不过了解数据对象的构成和规模信息还是值得的。</p>
<p>现在，我们有了数据，就用<code>SGDRegressor</code>来拟合：</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [13]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">from</span> <span class="nn">sklearn</span> <span class="k">import</span> <span class="n">linear_model</span>
<span class="n">sgd</span> <span class="o">=</span> <span class="n">linear_model</span><span class="o">.</span><span class="n">SGDRegressor</span><span class="p">()</span>
<span class="n">train</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">([</span><span class="kc">True</span><span class="p">,</span> <span class="kc">False</span><span class="p">],</span> <span class="n">size</span><span class="o">=</span><span class="nb">len</span><span class="p">(</span><span class="n">y</span><span class="p">),</span> <span class="n">p</span><span class="o">=</span><span class="p">[</span><span class="o">.</span><span class="mi">75</span><span class="p">,</span> <span class="o">.</span><span class="mi">25</span><span class="p">])</span>
<span class="n">sgd</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="n">train</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="n">train</span><span class="p">])</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">
<div class="prompt output_prompt">Out[13]:</div>


<div class="output_text output_subarea output_execute_result">
<pre>SGDRegressor(alpha=0.0001, average=False, epsilon=0.1, eta0=0.01,
       fit_intercept=True, l1_ratio=0.15, learning_rate='invscaling',
       loss='squared_loss', n_iter=5, penalty='l2', power_t=0.25,
       random_state=None, shuffle=True, verbose=0, warm_start=False)</pre>
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>这里又出现一个“充实的（beefy）”对象。重点需要了解我们的损失函数是<code>squared_loss</code>，与线性回归里的残差平方和是一样的。还需要注意<code>shuffle</code>会对数据产生随机搅动（shuffle），这在解决伪相关问题时很有用。scikit-learn用<code>fit_intercept</code>方法可以自动加一列1。如果想从拟合结果中看到很多输出，就把<code>verbose</code>设为1。用scikit-learn的API预测，我们可以统计残差的分布情况：</p>

</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [16]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">linear_preds</span> <span class="o">=</span> <span class="n">sgd</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X</span><span class="p">[</span><span class="o">~</span><span class="n">train</span><span class="p">])</span>
</pre></div>

</div>
</div>
</div>

</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="prompt input_prompt">In [17]:</div>
<div class="inner_cell">
    <div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="o">%</span><span class="k">matplotlib</span> inline
<span class="kn">from</span> <span class="nn">matplotlib</span> <span class="k">import</span> <span class="n">pyplot</span> <span class="k">as</span> <span class="n">plt</span>
<span class="n">f</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">7</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
<span class="n">f</span><span class="o">.</span><span class="n">tight_layout</span><span class="p">()</span>
<span class="n">ax</span><span class="o">.</span><span class="n">hist</span><span class="p">(</span><span class="n">linear_preds</span> <span class="o">-</span> <span class="n">y</span><span class="p">[</span><span class="o">~</span><span class="n">train</span><span class="p">],</span><span class="n">label</span><span class="o">=</span><span class="s1">'Residuals Linear'</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s1">'b'</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=.</span><span class="mi">5</span><span class="p">);</span>
<span class="n">ax</span><span class="o">.</span><span class="n">set_title</span><span class="p">(</span><span class="s2">"Residuals"</span><span class="p">)</span>
<span class="n">ax</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="s1">'best'</span><span class="p">);</span>
</pre></div>

</div>
</div>
</div>

<div class="output_wrapper">
<div class="output">


<div class="output_area">
<div class="prompt"></div>


<div class="output_png output_subarea ">
<img src="%0AAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X2cnWV97/vPN4k8P4QYDQQSghq2jcWNYo371MooGFNq%0AgS3Kg8oJ3dl292QfKu3eFagenRxsBFvloVtoT6USUBAsFVAQk6Ij6G6YimApMQLaUBJMooEkiDUk%0A8Dt/rDtxZTLJTOYhM4TP+/Var7nX777ua13XzCTru+6nSVUhSZJe3MaM9AAkSdLIMxBIkiQDgSRJ%0AMhBIkiQMBJIkCQOBJEnCQCCpF0nel+TrO1nflWTuELxOR5LHB9uPpMEzEEh7gCTLk/wiydNJViW5%0ALslBA+2vqr5QVe/YWZPmIWkPYSCQ9gwFvLOqDgT+I3AM8JGRHZKkFxIDgbSHqarVwCLgNQBJ3pTk%0Afyd5KskDSY7f0jbJOUl+lGRDkh8neW9b/Z62dm9PsizJuiR/CaRtXWeS69qeT0vyfJIxzfPfS7K0%0AeY0fJfn9HY09yflJVjRtlyV52xB+ayTthIFA2nMEIMkRwGzg3iSHA18F/t+qOgT4n8DNSV6aZH/g%0AcmB2VR0E/Cfgge06TSYCNwN/CrwU+BHwm21N+jp0sBr4neY1fg+4NMnrenmd/wD8d+ANTdtZwPJ+%0Azl3SIBkIpD1DgFuSbAD+jdab9p8B7wfuqKo7AarqH4DvAr9D6438eeCYJPtW1eqqWtpL3ycB/1JV%0Af19Vz1XVZcCqHq+9Q1V1R1X9a7N8N629F7/VS9PngL2B1yR5SVX9W1X9uL/fAEmDYyCQ9gwFnNJ8%0Asu4A3gYcBxwJvKc5XPBUkqdofbo/tKp+AZwB/AHwRJKvNp/Se5oMrOhR6/eVAUl+O8mSJGub1z+J%0A1p6GbSdQ9ShwHtAJrE5yQ5LD+vs6kgbHQCDtYZpP4X8JXEJrb8F1VXVI2+PAqvpk03ZRVc0CDgWW%0AAX/TS5dPAFO2PEmS9ufAz4H92p4f2tZ2b1qHGz4JvLw5bHEHO9irUFU3VNVv0Qoy1cxB0m5gIJD2%0ATJcBbwS+DfxukllJxibZp7n2//AkL09ySnMuwSbgGVq77Xu6g9Zu/P+cZBzwh7S96dM67+AtSaYk%0AORi4sG3dXs3jZ8DzSX6b1rkB20lydJK3NSFiI/DLHYxH0jAwEEh7oKr6GbAQ+GPgZFonBK6htcfg%0Af9D6hD4G+CNgJbCW1nH9/2tLF81jS1/vAS6m9cb+KlpBY8tr/QNwI/DPwD8BX2nb9mlaAeIm4Eng%0ALODWnsNtvu4NfAL4KfATYCLbhgtJwyhVOz9BOMkHgf9K6z+Qv6mqy5NMoPUfwJG0zgI+varWNe0v%0ABP4LrWT/h1W1qKkfB1wD7EPrJKcPNvW9gWuB19P6T+mMqnpsaKcpSZJ2Zqd7CJL8Oq0w8Bu0bnby%0AziSvBC4AFlfV0cBdzXOSzKB1ktIMWpc9XdkcbwS4CphbVdOB6UlmN/W5wNqmfikeM5Qkabfr65DB%0Aq4F7q+qXVfUc8C3gNFq7IBc2bRYCpzbLpwA3VNWmqloOPArMbM4UPrCqupt217Zt097XzcAJg5uS%0AJEnaVX0Fgn8BfivJhCT70bpc6AhgUnM3NGjddGRSs9zz8qQVwOG91Fc2dZqvjwNU1WZgfXNIQpIk%0A7SbjdrayqpYluYTWjUSeoXU28XM92lQS/8iJJEkvYDsNBABV9bfA3wIk+TNan/RXJzm0qlY1hwPW%0ANM1Xsu31yUc07Vc2yz3rW7aZSuvGKOOAg6vqyZ7jMHRIktS7qtrpHUP7o8/LDpO8vPk6FXgXcD1w%0AGzCnaTIHuKVZvg04M8leSY4CpgPdVbUK2JBkZnOS4dn86tKj9r7eTeskxV5V1R79+NjHPjbiY3CO%0AztE5Osc96fFimONQ6XMPAfB3SV5K68Yl86pqfZKLgZuSzKW57LB5w16a5CZgKbC5ab9ltPNoXXa4%0AL233VgeuBq5L8gityw7PHJKZSZKkfuvPIYO39FJ7EjhxB+0XAAt6qd9H62+096xvpAkUkiRpZHin%0AwlGko6NjpIcw7JzjnsE57hmco9r1eafC0SJJvVDGKknS7pKEGoKTCvtzDoEkaQj96gau0q4Zzg/G%0ABgJJGgHu8dSuGu4g6TkEkiTJQCBJkgwEkiQJA4EkaTf4whe+wDve8Y4dru/o6ODqq68e9Ot0dXUx%0AZcqUvhvuggMPPJDly5cPaZ+jkScVStIocN55naxbN3z9jx8Pl13W2a+206ZNY82aNYwdO5b999+f%0At7/97XzmM5/hoIMOGvDrv+997+N973vfDtcnGdGrL8455xymTJnCRRddtN26p59+egRGtPsZCCTt%0A1HC/UQ3UrrzBvRCsWwfTpnUOW//Ll/e/7yR89atf5W1vexurV6/mHe94Bx//+Mf55Cc/OWzjG2kj%0AHUh62rx5M+PG7d63aAOBpJ0a7jeqgdqVNzgN3KRJk5g1axYPPfTQ1tqSJUv44z/+Y37wgx9w5JFH%0Acvnll3P88ccDcM0113DRRRfx05/+lIkTJ/Lxj3+c9773vVxzzTVcffXV3HPPPQAsXryYc889l1Wr%0AVnH22WdvcxlmZ2cnP/rRj7juuusAWL58Oa94xSvYvHkzY8aM4XOf+xx//ud/zooVK3jZy17G+eef%0Az+///u/3Ov5LLrmEv/zLv2TDhg1MnjyZK6+8kre97W29tt3RpaBjxozh0Ucf5RWveAXnnHMO+++/%0AP4899hh33303M2bM4Prrr+cVr3gFAMuWLePcc8/le9/7Hi972cu46KKLeM973gPA7bffzkc+8hF+%0A/OMfc/DBBzN37lw+9rGPbTPHz372s8yfP5+jjjqKrq6u/v6YhoTnEEiStrPlzXHFihXceeedzJw5%0AE4CVK1fyzne+k49+9KM89dRT/MVf/AWnnXYaa9eu5ZlnnuGDH/wgd955Jxs2bOAf//EfOfbYY7fr%0A+2c/+xmnnXYaCxYsYO3atbzyla/kO9/5ztb1fX1SnzRpErfffjsbNmzgc5/7HH/0R3/E/fffv127%0AH/7wh3zmM5/hu9/9Lhs2bGDRokVMmzZtEN+VlhtvvJHOzk6eeuopXvWqV/HhD38YgGeeeYa3v/3t%0AvP/97+enP/0pX/ziF5k3bx4/+MEPADjggAP4/Oc/z/r167n99tu56qqruPXWW7fp++6772bZsmV8%0A/etfH/Q4d5WBQJK0jari1FNP5aCDDmLq1Km88pWv5CMf+QgAn//85znppJOYPXs2ACeeeCJveMMb%0AuP3220nCmDFjePDBB/n3f/93Jk2axIwZM7br/4477uDXf/3Xede73sXYsWM577zzOPTQQ7d5/Z05%0A6aSTOOqoowB4y1vewqxZs7bueWg3duxYNm7cyEMPPcSmTZuYOnXq1k/yA5WEd73rXbzhDW9g7Nix%0AvO997+OBBx4A4Ktf/SpHHXUUc+bMYcyYMRx77LG8613v4ktf+hIAxx9/PK95zWsAOOaYYzjzzDP5%0A1re+tU3/nZ2d7Lvvvuy9996DGudAGAgkSdtIwq233sqGDRvo6uriG9/4Bt/97ncBeOyxx/jSl77E%0AIYccsvXxne98h1WrVrHffvtx44038ld/9VdMnjyZd77znfzwhz/crv8nnniCI444YpvarlwZ8LWv%0AfY03velNvPSlL+WQQw7hjjvuYO3atdu1e9WrXsVll11GZ2cnkyZN4qyzzuInP/nJLn43tjdp0qSt%0Ay/vuuy8///nPgdb35t57793me3P99dezevVqAO69917e+ta38vKXv5zx48fz13/919uNe6ivkNgV%0ABgJJ0g695S1v4dxzz+X8888HYOrUqZx99tk89dRTWx9PP/00H/rQhwCYNWsWixYtYtWqVbz61a/m%0AAx/4wHZ9Tp48mccff3zr86ra5vkBBxzAL37xi63PV61atXV548aNnHbaaXzoQx9izZo1PPXUU5x0%0A0kk73Ktw1llncc899/DYY4+RZOs8ejPYkwqnTp3K8ccfv9335jOf+QwA733vezn11FNZsWIF69at%0A4w/+4A94/vnnh3QMg2EgkCTt1HnnnUd3dzf33nsv73//+/nKV77CokWLeO655/jlL39JV1cXK1eu%0AZM2aNdx6660888wzvOQlL2H//fdn7Nix2/V30kkn8dBDD/HlL3+ZzZs3c8UVV2zzpn/sscdy9913%0A8/jjj7N+/Xo+8YlPbF337LPP8uyzzzJx4kTGjBnD1772NRYtWtTruB9++GG+8Y1vsHHjRvbee2/2%0A2WefXscDrVCyefNmfvnLX259bNq0qdd2O/I7v/M7PPzww3z+859n06ZNbNq0iX/6p39i2bJlAPz8%0A5z/nkEMOYa+99qK7u5vrr79+VF3ZYCCQJO3UxIkTmTNnDpdccglHHHEEt956KwsWLODlL385U6dO%0A5VOf+hRVxfPPP8+ll17K4Ycfzktf+lLuuecerrrqKmDby/omTpzIl770JS644AImTpzIo48+ypvf%0A/Oatr3fiiSdyxhln8NrXvpbf+I3f4Hd/93e3bnvggQdyxRVXcPrppzNhwgRuuOEGTjnllG3Gu6Xt%0Axo0bufDCC3nZy17GYYcdxs9+9rNtwkXPbS6++GL222+/rY8TTjih13Y938Tbx7Zo0SK++MUvcvjh%0Ah3PYYYdx4YUX8uyzzwJw5ZVX8tGPfpSDDjqIiy66iDPOOKPXfkZKXih/cStJvVDGKu1Jzjmnc9Re%0AdnjNNZ0jPYwBaf5+/Ta10XRjIo1Ovf3etNUHnSa8D4EkjQK+WWukechAkiQZCCRJkoFAkiRhIJAk%0ASRgIJEkSBgJJkoSXHUrSiBjpm9BIPRkIJGk38yZrGo36PGSQ5MIkDyV5MMn1SfZOMiHJ4iQPJ1mU%0AZHyP9o8kWZZkVlv9uKaPR5Jc3lbfO8mNTX1JkiOHfpqSJGlndhoIkkwDPgC8vqqOAcYCZwIXAIur%0A6mjgruY5SWYAZwAzgNnAlfnVfrGrgLlVNR2YnmR2U58LrG3qlwKXDNnsJElSv/S1h2ADsAnYL8k4%0AYD/gCeBkYGHTZiFwarN8CnBDVW2qquXAo8DMJIcBB1ZVd9Pu2rZt2vu6Gdj+r0lIkqRhtdNAUFVP%0AAp8C/o1WEFhXVYuBSVW1umm2GpjULE8GVrR1sQI4vJf6yqZO8/Xx5vU2A+uTTBjohCRJ0q7r65DB%0AK4HzgGm03tQPSPL+9jbNnyD0DBlJkl7A+rrK4A3A/66qtQBJ/h74T8CqJIdW1armcMCapv1KYErb%0A9kfQ2jOwslnuWd+yzVTgieawxMHNnontdHZ2bl3u6Oigo6Ojr/lJkrRH6erqoqura8j77SsQLAP+%0AnyT7Ar8ETgS6gWeAObROAJwD3NK0vw24PsmnaR0KmA50V1Ul2ZBkZrP92cAVbdvMAZYA76Z1kmKv%0A2gOBJEkvRj0/EM+fP39I+t1pIKiq7ye5Fvgu8DzwPeD/Aw4EbkoyF1gOnN60X5rkJmApsBmYV7+6%0A4HYecA2wL3BHVd3Z1K8GrkvyCLCW1lUMkiRpN+rzxkRV9Ungkz3KT9LaW9Bb+wXAgl7q9wHH9FLf%0ASBMoJEnSyPBvGUiSJAOBJEkyEEiSJAwEkiQJA4EkScJAIEmSMBBIkiT6cR8CScPvvPM6WbdupEfR%0Au+7uB5g2baRHIWm4GQikUWDdOpg2rXOkh9Grb3/71L4bSXrB85CBJEkyEEiSJAOBJEnCQCBJkjAQ%0ASJIkDASSJAkDgSRJwkAgSZIwEEiSJAwEkiQJA4EkScJAIEmSMBBIkiQMBJIkCQOBJEnCQCBJkjAQ%0ASJIkDASSJIl+BIIk/yHJ/W2P9Un+MMmEJIuTPJxkUZLxbdtcmOSRJMuSzGqrH5fkwWbd5W31vZPc%0A2NSXJDly6KcqSZJ2pM9AUFU/rKrXVdXrgOOAXwBfBi4AFlfV0cBdzXOSzADOAGYAs4Erk6Tp7ipg%0AblVNB6Ynmd3U5wJrm/qlwCVDNUFJktS3XT1kcCLwaFU9DpwMLGzqC4FTm+VTgBuqalNVLQceBWYm%0AOQw4sKq6m3bXtm3T3tfNwAm7OhFJkjRwuxoIzgRuaJYnVdXqZnk1MKlZngysaNtmBXB4L/WVTZ3m%0A6+MAVbUZWJ9kwi6OTZIkDdC4/jZMshfwu8D5PddVVSWpoRxYbzo7O7cud3R00NHRMdwvKUnSqNLV%0A1UVXV9eQ99vvQAD8NnBfVf20eb46yaFVtao5HLCmqa8EprRtdwStPQMrm+We9S3bTAWeSDIOOLiq%0Anuw5gPZAIEnSi1HPD8Tz588fkn535ZDBWfzqcAHAbcCcZnkOcEtb/cwkeyU5CpgOdFfVKmBDkpnN%0ASYZnA7f20te7aZ2kKEmSdpN+7SFIsj+tEwo/0Fa+GLgpyVxgOXA6QFUtTXITsBTYDMyrqi2HE+YB%0A1wD7AndU1Z1N/WrguiSPAGtpnasgSZJ2k34Fgqp6BpjYo/YkrZDQW/sFwIJe6vcBx/RS30gTKCRJ%0A0u7nnQolSZKBQJIkGQgkSRIGAkmShIFAkiRhIJAkSRgIJEkSBgJJkoSBQJIkYSCQJEkYCCRJEgYC%0ASZKEgUCSJGEgkCRJGAgkSRIGAkmShIFAkiRhIJAkSRgIJEkSBgJJkoSBQJIkYSCQJEkYCCRJEgYC%0ASZKEgUCSJGEgkCRJ9DMQJBmf5O+S/CDJ0iQzk0xIsjjJw0kWJRnf1v7CJI8kWZZkVlv9uCQPNusu%0Ab6vvneTGpr4kyZFDO01JkrQz/d1DcDlwR1X9GvBaYBlwAbC4qo4G7mqek2QGcAYwA5gNXJkkTT9X%0AAXOrajowPcnspj4XWNvULwUuGfTMJElSv/UZCJIcDPxWVf0tQFVtrqr1wMnAwqbZQuDUZvkU4Iaq%0A2lRVy4FHgZlJDgMOrKrupt21bdu093UzcMKgZiVJknZJf/YQHAX8NMnnknwvyd8k2R+YVFWrmzar%0AgUnN8mRgRdv2K4DDe6mvbOo0Xx+HVuAA1ieZMJAJSZKkXdefQDAOeD1wZVW9HniG5vDAFlVVQA39%0A8CRJ0u4wrh9tVgArquqfmud/B1wIrEpyaFWtag4HrGnWrwSmtG1/RNPHyma5Z33LNlOBJ5KMAw6u%0Aqid7DqSzs3PrckdHBx0dHf0YviRJe46uri66urqGvN8+A0Hzhv94kqOr6mHgROCh5jGH1gmAc4Bb%0Amk1uA65P8mlahwKmA91VVUk2JJkJdANnA1e0bTMHWAK8m9ZJittpDwSSJL0Y9fxAPH/+/CHptz97%0ACADOBb6QZC/gR8DvAWOBm5LMBZYDpwNU1dIkNwFLgc3AvOaQAsA84BpgX1pXLdzZ1K8GrkvyCLAW%0AOHOQ85IkSbugX4Ggqr4P/EYvq07cQfsFwIJe6vcBx/RS30gTKCRJ0u7nnQolSZKBQJIkGQgkSRIG%0AAkmShIFAkiRhIJAkSRgIJEkSBgJJkoSBQJIkYSCQJEkYCCRJEgYCSZKEgUCSJGEgkCRJGAgkSRIG%0AAkmShIFAkiRhIJAkSRgIJEkSBgJJkoSBQJIkYSCQJEkYCCRJEgYCSZKEgUCSJGEgkCRJGAgkSRL9%0ADARJlif55yT3J+luahOSLE7ycJJFSca3tb8wySNJliWZ1VY/LsmDzbrL2+p7J7mxqS9JcuRQTlKS%0AJO1cf/cQFNBRVa+rqjc2tQuAxVV1NHBX85wkM4AzgBnAbODKJGm2uQqYW1XTgelJZjf1ucDapn4p%0AcMkg5yVJknbBrhwySI/nJwMLm+WFwKnN8inADVW1qaqWA48CM5McBhxYVd1Nu2vbtmnv62bghF0Y%0AlyRJGqRd2UPwD0m+m+QDTW1SVa1ullcDk5rlycCKtm1XAIf3Ul/Z1Gm+Pg5QVZuB9Ukm7MpEJEnS%0AwI3rZ7vfrKqfJHkZsDjJsvaVVVVJauiHJ0mSdod+BYKq+knz9adJvgy8EVid5NCqWtUcDljTNF8J%0ATGnb/AhaewZWNss961u2mQo8kWQccHBVPdlzHJ2dnVuXOzo66Ojo6M/wJUnaY3R1ddHV1TXk/fYZ%0ACJLsB4ytqqeT7A/MAuYDtwFzaJ0AOAe4pdnkNuD6JJ+mdShgOtDd7EXYkGQm0A2cDVzRts0cYAnw%0AblonKW6nPRBIkvRi1PMD8fz584ek3/7sIZgEfLm5UGAc8IWqWpTku8BNSeYCy4HTAapqaZKbgKXA%0AZmBeVW05nDAPuAbYF7ijqu5s6lcD1yV5BFgLnDkEc5MkSf3UZyCoqn8Fju2l/iRw4g62WQAs6KV+%0AH3BML/WNNIFCkiTtft6pUJIkGQgkSZKBQJIkYSCQJEkYCCRJEgYCSZKEgUCSJGEgkCRJGAgkSRIG%0AAkmShIFAkiRhIJAkSRgIJEkSBgJJkoSBQJIkYSCQJEkYCCRJEgYCSZKEgUCSJGEgkCRJGAgkSRIG%0AAkmShIFAkiRhIJAkScC4kR6AJA1Ed/cSzjmnc6SH0avx4+GyyzpHehjSLjEQSHpBevbZfZg2rXOk%0Ah9Gr5cs7R3oI0i7r1yGDJGOT3J/kK83zCUkWJ3k4yaIk49vaXpjkkSTLksxqqx+X5MFm3eVt9b2T%0A3NjUlyQ5cignKEmS+tbfcwg+CCwFqnl+AbC4qo4G7mqek2QGcAYwA5gNXJkkzTZXAXOrajowPcns%0Apj4XWNvULwUuGdyUJEnSruozECQ5AjgJ+Cyw5c39ZGBhs7wQOLVZPgW4oao2VdVy4FFgZpLDgAOr%0Aqrtpd23bNu193QycMODZSJKkAenPHoJLgT8Bnm+rTaqq1c3yamBSszwZWNHWbgVweC/1lU2d5uvj%0AAFW1GVifZMIuzEGSJA3STgNBkncCa6rqfn61d2AbVVX86lCCJEl6AerrKoP/Azg5yUnAPsBBSa4D%0AVic5tKpWNYcD1jTtVwJT2rY/gtaegZXNcs/6lm2mAk8kGQccXFVP9jaYzs7OrcsdHR10dHT0OUFJ%0AkvYkXV1ddHV1DXm/Ow0EVfWnwJ8CJDke+J9VdXaSTwJzaJ0AOAe4pdnkNuD6JJ+mdShgOtBdVZVk%0AQ5KZQDdwNnBF2zZzgCXAu2mdpNir9kAgSdKLUc8PxPPnzx+Sfnf1PgRbDg1cDNyUZC6wHDgdoKqW%0AJrmJ1hUJm4F5zSEFgHnANcC+wB1VdWdTvxq4LskjwFrgzIFNRZIkDVS/A0FVfQv4VrP8JHDiDtot%0AABb0Ur8POKaX+kaaQCFJkkaGf8tAkiQZCCRJkoFAkiRhIJAkSRgIJEkSBgJJkoSBQJIkYSCQJEkY%0ACCRJEgYCSZKEgUCSJGEgkCRJGAgkSRIGAkmShIFAkiRhIJAkSRgIJEkSBgJJkoSBQJIkYSCQJEkY%0ACCRJEgYCSZKEgUCSJGEgkCRJGAgkSRIGAkmShIFAkiTRRyBIsk+Se5M8kGRpkk809QlJFid5OMmi%0AJOPbtrkwySNJliWZ1VY/LsmDzbrL2+p7J7mxqS9JcuRwTFSSJO3YTgNBVf0SeGtVHQu8FnhrkjcD%0AFwCLq+po4K7mOUlmAGcAM4DZwJVJ0nR3FTC3qqYD05PMbupzgbVN/VLgkqGcoCRJ6lufhwyq6hfN%0A4l7AWOAp4GRgYVNfCJzaLJ8C3FBVm6pqOfAoMDPJYcCBVdXdtLu2bZv2vm4GThjwbCRJ0oD0GQiS%0AjEnyALAa+GZVPQRMqqrVTZPVwKRmeTKwom3zFcDhvdRXNnWar48DVNVmYH2SCQObjiRJGohxfTWo%0AqueBY5McDHw9yVt7rK8kNVwDlCRJw6/PQLBFVa1PcjtwHLA6yaFVtao5HLCmabYSmNK22RG09gys%0AbJZ71rdsMxV4Isk44OCqerK3MXR2dm5d7ujooKOjo7/DlzjvvE7WrRvpUfSuu/sBpk0b6VFIeiHo%0A6uqiq6tryPvdaSBIMhHYXFXrkuwLvB2YD9wGzKF1AuAc4JZmk9uA65N8mtahgOlAd7MXYUOSmUA3%0AcDZwRds2c4AlwLtpnaTYq/ZAIO2qdetg2rTOkR5Gr7797VP7biRJbP+BeP78+UPSb197CA4DFiYZ%0AQ+t8g+uq6q4k9wM3JZkLLAdOB6iqpUluApYCm4F5VbXlcMI84BpgX+COqrqzqV8NXJfkEWAtcOaQ%0AzEySJPXbTgNBVT0IvL6X+pPAiTvYZgGwoJf6fcAxvdQ30gQKSZI0MrxToSRJMhBIkiQDgSRJwkAg%0ASZIwEEiSJAwEkiQJA4EkScJAIEmSMBBIkiQMBJIkCQOBJEnCQCBJkjAQSJIkDASSJAkDgSRJwkAg%0ASZIwEEiSJAwEkiQJA4EkScJAIEmSMBBIkiQMBJIkCQOBJEnCQCBJkjAQSJIkDASSJAkDgSRJoh+B%0AIMmUJN9M8lCSf0nyh019QpLFSR5OsijJ+LZtLkzySJJlSWa11Y9L8mCz7vK2+t5JbmzqS5IcOdQT%0AlSRJO9afPQSbgD+qqtcAbwL+e5JfAy4AFlfV0cBdzXOSzADOAGYAs4Erk6Tp6ypgblVNB6Ynmd3U%0A5wJrm/qlwCVDMjtJktQvfQaCqlpVVQ80yz8HfgAcDpwMLGyaLQRObZZPAW6oqk1VtRx4FJiZ5DDg%0AwKrqbtpd27ZNe183AycMZlKSJGnX7NI5BEmmAa8D7gUmVdXqZtVqYFKzPBlY0bbZCloBomd9ZVOn%0A+fo4QFVtBtYnmbArY5MkSQM3rr8NkxxA69P7B6vq6V8dBYCqqiQ1DOPbRmdn59bljo4OOjo6hvsl%0AJUkaVbq6uujq6hryfvsVCJK8hFYYuK6qbmnKq5McWlWrmsMBa5r6SmBK2+ZH0NozsLJZ7lnfss1U%0A4Ikk44CDq+rJnuNoDwSSJL0Y9fxAPH/+/CHptz9XGQS4GlhaVZe1rboNmNMszwFuaaufmWSvJEcB%0A04HuqloFbEgys+nzbODWXvp6N62TFCVJ0m7Snz0Evwm8H/jnJPc3tQuBi4GbkswFlgOnA1TV0iQ3%0AAUuBzcC8qtpyOGEecA2wL3BHVd3Z1K8GrkvyCLAWOHOQ85IkSbugz0BQVd9mx3sSTtzBNguABb3U%0A7wOO6aW+kSZQSJKk3c87FUqSJAOBJEkyEEiSJAwEkiQJA4EkScJAIEmSMBBIkiQMBJIkCQOBJEnC%0AQCBJkjCb/n6IAAAJOUlEQVQQSJIkDASSJAkDgSRJwkAgSZIwEEiSJAwEkiQJA4EkScJAIEmSMBBI%0AkiQMBJIkCQOBJEnCQCBJkjAQSJIkDASSJAkDgSRJwkAgSZLoRyBI8rdJVid5sK02IcniJA8nWZRk%0AfNu6C5M8kmRZkllt9eOSPNisu7ytvneSG5v6kiRHDuUEJUlS3/qzh+BzwOwetQuAxVV1NHBX85wk%0AM4AzgBnNNlcmSbPNVcDcqpoOTE+ypc+5wNqmfilwySDmI0mSBqDPQFBV9wBP9SifDCxslhcCpzbL%0ApwA3VNWmqloOPArMTHIYcGBVdTftrm3bpr2vm4ETBjAPSZI0CAM9h2BSVa1ullcDk5rlycCKtnYr%0AgMN7qa9s6jRfHweoqs3A+iQTBjguSZI0AIM+qbCqCqghGIskSRoh4wa43eokh1bVquZwwJqmvhKY%0A0tbuCFp7BlY2yz3rW7aZCjyRZBxwcFU92duLdnZ2bl3u6Oigo6NjgMOXJOmFqauri66uriHvd6CB%0A4DZgDq0TAOcAt7TVr0/yaVqHAqYD3VVVSTYkmQl0A2cDV/ToawnwblonKfaqPRBIkvRi1PMD8fz5%0A84ek3z4DQZIbgOOBiUkeBz4KXAzclGQusBw4HaCqlia5CVgKbAbmNYcUAOYB1wD7AndU1Z1N/Wrg%0AuiSPAGuBM4dkZpIkqd/6DARVddYOVp24g/YLgAW91O8DjumlvpEmUEjSnqC7ewnnnNM50sPYzvjx%0AcNllnSM9DI1SAz1kIEnagWef3Ydp0zpHehjbWb68c6SHoFHMWxdLkiQDgSRJMhBIkiQMBJIkCQOB%0AJEnCQCBJkjAQSJIkDASSJAkDgSRJwkAgSZIwEEiSJAwEkiQJA4EkScK/dqhhcN55naxbN9Kj2F53%0A9wNMmzbSo5Ck0clAoCG3bh2j8k+/fvvbp470ECRp1PKQgSRJMhBIkiQDgSRJwkAgSZIwEEiSJAwE%0AkiQJA4EkScJAIEmSMBBIkiQMBJIkCQOBJEliFAWCJLOTLEvySJLzR3o8kiS9mIyKQJBkLPC/gNnA%0ADOCsJL82sqPa/bq6ukZ6CMNu+fKukR7CsHOOe4YXwxxfDP/nvBjmOFRGy187fCPwaFUtB0jyReAU%0A4AcjOajdrauri46OjpEexrBavryLadM6RnoYw8o57hn2xDl2dy/hnHM6tz5/4IEujj22Y8TG0278%0AeLjsss4h7/fF8P/qUBktgeBw4PG25yuAmSM0lheE887rZN26kR5F77q7H2DatJEehaSenn12n23+%0ANPny5Z2j5k+VL1/eOdJDeNEbLYGgRnoAPVUVn/rUp1m//und9prf+lYXzz3X2a+2Dz+8hje+8crh%0AHdAAffvbp470ECRJuyhVI/9enORNQGdVzW6eXwg8X1WXtLUZ+YFKkjQKVVUG28doCQTjgB8CJwBP%0AAN3AWVX1ojqHQJKkkTIqDhlU1eYk/zfwdWAscLVhQJKk3WdU7CGQJEkja1TchwAgyYQki5M8nGRR%0AkvE7aLfTGxgl+R9Jnk8yYfhHvesGO88kFyX5fpIHktyVZMruG33/DMEc/zzJD5p5/n2Sg3ff6Ptn%0ACOb4niQPJXkuyet338j71p+bhCW5oln//SSv25VtR4NBzvFvk6xO8uDuG/GuG+gck0xJ8s3m9/Nf%0Akvzh7h15/w1ijvskubf5f3Rpkk/s3pH332B+V5t1Y5Pcn+Qrfb5YVY2KB/BJ4EPN8vnAxb20GQs8%0ACkwDXgI8APxa2/opwJ3AvwITRnpOwzFP4MC2ducCnx3pOQ3DHN8OjGmWL+5t+5F+DMEcXw0cDXwT%0AeP1Iz6c/Y25rcxJwR7M8E1jS321Hw2Mwc2ye/xbwOuDBkZ7LMP0cDwWObZYPoHV+1574c9yv+ToO%0AWAK8eaTnNNRzbGp/DHwBuK2v1xs1ewiAk4GFzfJCoLdr17bewKiqNgFbbmC0xaeBDw3rKAdvUPOs%0AqvbrIA8AfjaMYx2owc5xcVU937S7FzhimMc7EIOd47Kqeni3jHTX9PVvDNrmXlX3AuOTHNrPbUeD%0AwcyRqroHeGo3jncgBjrHSVW1qqoeaOo/p3WDuMm7b+j9NuA5Ns9/0bTZi9Yb75O7ZdS7ZlBzTHIE%0ArcDwWaDPqxBGUyCYVFWrm+XVwKRe2vR2A6PDAZKcAqyoqn8e1lEO3qDmCZDkz5L8GzCH1ifo0WbQ%0Ac2zzX4A7hnZ4Q2Io5zia9GfMO2ozuR/bjgaDmeMLxUDnuE34TjKN1t6Qe4d8hIM3qDk2u9IfoPXv%0A95tVtXQYxzpQg/1dvRT4E+B5+mG3XmWQZDGt3VE9fbj9SVVVer/vQK9nQCbZF/hTWruat5YHOs7B%0AGq55tm33YeDDSS6g9QP/vYGOdaCGe47Na3wYeLaqrh/YKAdnd8xxFOrvmEfs39cQGOgcX0g/z0HP%0AMckBwN8BH2z2FIw2g5pjVT0HHNuco/T1JB1V1TWE4xsKA51jkrwTWFNV9yfp6E8nuzUQVNXbd7Su%0AOUnn0KpaleQwYE0vzVbSOk9giym00tAraR1j+X4SaCXA+5K8sap662dYDeM8e7qeEfr0PNxzTHIO%0ArV1dJwzNiHfdbvw5jib9GXPPNkc0bV7Sj21Hg4HOceUwj2soDWqOSV4C3Ax8vqpuGcZxDsaQ/Byr%0Aan2S24E3AF1DP8xBGcwcTwNOTnISsA9wUJJrq+r/3OGrjfRJE20nPnwSOL9ZvoDeT9IaB/yI1pv/%0AXuzgpCVG/0mFA54nML2t3bnAdSM9p2GY42zgIWDiSM9luObY1uabwHEjPZ9dHHP7SUxv4lcno/Xr%0A3+dIPwYzx7b10xjdJxUO5ucY4Frg0pGexzDOcSIwvlneF7gbOGGk5zSUc+zR5njgK32+3khPuG3A%0AE4B/AB4GFrX9sCYDt7e1+21aZ70+Cly4g75+zOgNBIOaJ61deA82vxg3Ay8f6TkNwxwfAR4D7m8e%0AV470nIZhjv+Z1nG/fwdWAV8b6TntbMzAfwP+W1ub/9Ws/z5tV0n059/naHgMco430Lqj6sbmZ/h7%0AIz2foZwj8GZax5wfaPs3OHuk5zPEczwG+F4zx38G/mSk5zIcv6tt64+nH1cZeGMiSZI0qq4ykCRJ%0AI8RAIEmSDASSJMlAIEmSMBBIkiQMBJIkCQOBJEnCQCBJkoD/H1gJgfFc52KhAAAAAElFTkSuQmCC">
</div>

</div>

</div>
</div>

</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>拟合的效果非常好。异常值很少，直方图也呈现出完美的正态分布钟形图。</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="How-it-works...">How it works...<a class="anchor-link" href="using-stochastic-gradient-descent-for-regression.html#How-it-works...">¶</a>
</h3>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered">
<div class="prompt input_prompt">
</div>
<div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>当然这里我们用的是虚拟数据集，但是你也可以用更大的数据集合。例如，如果你在华尔街工作，有可能每天一个市场都有20亿条交易数据。现在如果有一周或一年的数据，用SGD算法就可能无法运行了。很难处理这么大的数据量，因为标准的梯度下降法每一步都要计算梯度，计算量非常庞大。</p>
<p>标准的梯度下降法的思想是在每次迭代计算一个新的相关系数矩阵，然后用学习速率（learning rate）和目标函数（objective function）的梯度调整它，直到相关系数矩阵收敛为止。如果用伪代码写就是这样：</p>

<pre><code>while not_converged:
    w = w – learning_rate * gradient(cost(w))</code></pre>
<p>这里涉及的变量包括：</p>
<ul>
<li>
<code>w</code>：相关系数矩阵</li>
<li>
<code>learning_rate</code>：每次迭代时前进的长度。如果收敛效果不好，调整这个参数很重要</li>
<li>
<code>gradient</code>：导数矩阵</li>
<li>
<code>cost</code>：回归的残差平方和。后面我们会介绍，不同的分类方法中损失函数定义不同，具有可变性也是SGD应用广泛的理由之一。</li>
</ul>
<p>除了梯度函数有点复杂之外，这个方法还是可以的。随着相关系数向量的增加，梯度的计算也会变得越来越慢。每次更新之前，我们都需要对每个数据点计算新权重。</p>
<p>SGD的工作方式稍有不同；每次迭代不是批量更新梯度，而是只更新新数据点的参数。这些数据点是随机选择的，因此称为随机梯度下降法。</p>

</div>
</div>
</div>
    </div>
  </div>

    </div>
    <aside class="postpromonav"><nav><ul itemprop="keywords" class="tags">
<li><a class="tag p-category" href="../categories/chs.html" rel="tag">CHS</a></li>
            <li><a class="tag p-category" href="../categories/ipython.html" rel="tag">ipython</a></li>
            <li><a class="tag p-category" href="../categories/machine-learning.html" rel="tag">Machine Learning</a></li>
            <li><a class="tag p-category" href="../categories/python.html" rel="tag">Python</a></li>
            <li><a class="tag p-category" href="../categories/scikit-learn-cookbook.html" rel="tag">scikit-learn cookbook</a></li>
        </ul>
<ul class="pager hidden-print">
<li class="previous">
                <a href="using-pipelines-for-multiple-preprocessing-steps.html" rel="prev" title="using-pipelines-for-multiple-preprocessing-steps">Previous post</a>
            </li>
            <li class="next">
                <a href="using-truncated-svd-to-reduce-dimensionality.html" rel="next" title="using-truncated-svd-to-reduce-dimensionality">Next post</a>
            </li>
        </ul></nav></aside><script type="text/javascript" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"> </script><script type="text/x-mathjax-config">
MathJax.Hub.Config({
    tex2jax: {
        inlineMath: [ ['$','$'], ["\\(","\\)"] ],
        displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
        processEscapes: true
    },
    displayAlign: 'center', // Change this to 'center' to center equations.
    "HTML-CSS": {
        styles: {'.MathJax_Display': {"margin": 0}}
    }
});
</script></article>
</div>
        <!--End of body content-->

        <footer id="footer">
            Contents © 2017         <a href="mailto:muxuezi@gmail.com">Tao Junjie</a> - Powered by         <a href="https://getnikola.com" rel="nofollow">Nikola</a>         
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0">
<img alt="Creative Commons License BY-NC-SA" style="border-width:0; margin-bottom:12px;" src="http://i.creativecommons.org/l/by-nc-sa/4.0/80x15.png"></a>
            
        </footer>
</div>
</div>


            <script src="../assets/js/all-nocdn.js"></script><script>$('a.image-reference:not(.islink) img:not(.islink)').parent().colorbox({rel:"gal",maxWidth:"100%",maxHeight:"100%",scalePhotos:true});</script><!-- fancy dates --><script>
    moment.locale("en");
    fancydates(0, "YYYY-MM-DD HH:mm");
    </script><!-- end fancy dates --><script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-51330059-1', 'auto');
  ga('send', 'pageview');

</script>
</body>
</html>
